The present invention relates, in general, to neural net architecture, and more particularly, to a neural net architecture which minimizes training time by enabling a feature map to recognize input signals that are rate variants of a previously learned signal pattern.
Advancements in neural net architecture have made neural nets the technology of choice for such advanced artificial intelligence applications as speech recognition and real-time handwriting recognition. Such advanced functions as speaker verification and signature verification may potentially be implemented using neural nets. There are, however, many problems to be overcome in these areas, not the least of which is the rate at which a speaker speaks, or a writer writes.
In the past, speech and handwriting recognition were approached in a number of ways. Dynamic Programming, described by Silverman, H. F., and Morgan, D.P., "The Application of Dynamic Programming to Connected Speech Recognition", IEEE ASSP Magazine, July 1990, pp 6-25, was a statistical approach which relied upon forward search with back-tracking to determine the probability that a given input corresponded to a certain pattern. Dynamic Programming was further refined using Hidden Markov Models as described by Picone, Joseph, "Continuous Speech Recognition Using Hidden Markov Models", IEEE ASSP Magazine, July 1990, pp 16-41. These were software implementations which required a long time to train to recognize varied inputs. Another approach was described by Tank, D. W., and Hopfield, "Concentrating Information in Time: Analog Neural Networks with Applications to Speech Recognition Problems", Procedures of the IEEE Conference on Neural Networks, San Diego, Jun. 21-24, 1987, pp IV455-IV468. Though this work demonstrated the applicability of neural nets to speech recognition, a practical application of the approach required a vast commitment of hardware. The pre-wired analog nets could only recognize the exact pattern for which they were wired. In order to overcome this shortcoming, additional circuitry for every possible variation of each input had to be added.
Advances in digital implementations of neural networks used less hardware than required by the Tank and Hopfield approach. However, the need for a feature map with a memorized pattern for each potential input limited the ability of the system to recognize variants in speaking rates. It was necessary to train the system to recognize each new input rate as it was encountered. This became very time consuming. Also, the feature map, and thus the memory requirements of the system, quickly multiplied to unwieldy proportions.